Known floating tool holders as disclosed, for example, in the M. L. Benjamin et al U.S. Pat. Nos. 3,421,770 and 3,454,283 are generally of relatively large diameter because the axial retainer for the floating connection is in the form of a threaded cap on the shank member surrounding the floating connection and having an in-turned flange overlying a radially out-turned flange of the floating drive member.